70 research outputs found
Answerer engagement in an enterprise social question & answering system
Many studies about social question and answer (Social Q&A) platforms have focused on the recommendation algorithms designed to find the right person for the question. But very little literature has investigated how to motivate the selected potential answerers to answer a question, especially in an enterprise setting. In this work, we designed an in-situ experiment in an enterprise social Q&A system to understand how different design aspects (e.g., exposing relationship information, directly asking the answerer, indicating question’s importance and urgency, and using virtual points as incentives) could influence answerers' engagement behaviors. We found that two design features examined in the experiment can affect answerers’ viewing and answering behaviors. These findings lead to specific design recommendations, which are also discussed in the paper
Does More Advice Help? The Effects of Second Opinions in AI-Assisted Decision Making
AI assistance in decision-making has become popular, yet people's
inappropriate reliance on AI often leads to unsatisfactory human-AI
collaboration performance. In this paper, through three pre-registered,
randomized human subject experiments, we explore whether and how the provision
of {second opinions} may affect decision-makers' behavior and performance in
AI-assisted decision-making. We find that if both the AI model's decision
recommendation and a second opinion are always presented together,
decision-makers reduce their over-reliance on AI while increase their
under-reliance on AI, regardless whether the second opinion is generated by a
peer or another AI model. However, if decision-makers have the control to
decide when to solicit a peer's second opinion, we find that their active
solicitations of second opinions have the potential to mitigate over-reliance
on AI without inducing increased under-reliance in some cases. We conclude by
discussing the implications of our findings for promoting effective human-AI
collaborations in decision-making
Malicious Selling Strategies During Livestream Shopping: A Case Study of Alibaba's Taobao and ByteDance's TikTok
Due to the limitations imposed by the COVID-19 pandemic, many users have
shifted their shopping patterns from offline to online. Livestream shopping has
become popular as one of the online shopping media. However, many streamers'
malicious selling behaviors have been reported. In this research, we sought to
explore streamers' malicious selling strategies and understand how viewers
perceive these strategies. First, we recorded 40 livestream shopping sessions
from two popular livestream platforms in China -- Taobao and TikTok (or
"Douyin" in Chinese). We identified four categories of malicious selling
strategies (i.e., Restrictive, Deceptive, Covert, and Asymmetric) and found
that platform designs enhanced these malicious selling strategies. Second,
through an interview study with 13 viewers, we provide a rich description of
viewers' awareness of malicious selling strategies and the challenges they
encountered while trying to overcome malicious selling. We conclude by
discussing the policy and design implications of countering malicious selling
'Don't Get Too Technical with Me': A Discourse Structure-Based Framework for Science Journalism
Science journalism refers to the task of reporting technical findings of a
scientific paper as a less technical news article to the general public
audience. We aim to design an automated system to support this real-world task
(i.e., automatic science journalism) by 1) introducing a newly-constructed and
real-world dataset (SciTechNews), with tuples of a publicly-available
scientific paper, its corresponding news article, and an expert-written short
summary snippet; 2) proposing a novel technical framework that integrates a
paper's discourse structure with its metadata to guide generation; and, 3)
demonstrating with extensive automatic and human experiments that our framework
outperforms other baseline methods (e.g. Alpaca and ChatGPT) in elaborating a
content plan meaningful for the target audience, simplifying the information
selected, and producing a coherent final report in a layman's style.Comment: Accepted to EMNLP 202
LLM-Powered Conversational Voice Assistants: Interaction Patterns, Opportunities, Challenges, and Design Guidelines
Conventional Voice Assistants (VAs) rely on traditional language models to
discern user intent and respond to their queries, leading to interactions that
often lack a broader contextual understanding, an area in which Large Language
Models (LLMs) excel. However, current LLMs are largely designed for text-based
interactions, thus making it unclear how user interactions will evolve if their
modality is changed to voice. In this work, we investigate whether LLMs can
enrich VA interactions via an exploratory study with participants (N=20) using
a ChatGPT-powered VA for three scenarios (medical self-diagnosis, creative
planning, and debate) with varied constraints, stakes, and objectivity. We
observe that LLM-powered VA elicits richer interaction patterns that vary
across tasks, showing its versatility. Notably, LLMs absorb the majority of VA
intent recognition failures. We additionally discuss the potential of
harnessing LLMs for more resilient and fluid user-VA interactions and provide
design guidelines for tailoring LLMs for voice assistance
Is a Seat at the Table Enough? Engaging Teachers and Students in Dataset Specification for ML in Education
Despite the promises of ML in education, its adoption in the classroom has
surfaced numerous issues regarding fairness, accountability, and transparency,
as well as concerns about data privacy and student consent. A root cause of
these issues is the lack of understanding of the complex dynamics of education,
including teacher-student interactions, collaborative learning, and classroom
environment. To overcome these challenges and fully utilize the potential of ML
in education, software practitioners need to work closely with educators and
students to fully understand the context of the data (the backbone of ML
applications) and collaboratively define the ML data specifications. To gain a
deeper understanding of such a collaborative process, we conduct ten co-design
sessions with ML software practitioners, educators, and students. In the
sessions, teachers and students work with ML engineers, UX designers, and legal
practitioners to define dataset characteristics for a given ML application. We
find that stakeholders contextualize data based on their domain and procedural
knowledge, proactively design data requirements to mitigate downstream harms
and data reliability concerns, and exhibit role-based collaborative strategies
and contribution patterns. Further, we find that beyond a seat at the table,
meaningful stakeholder participation in ML requires structured supports:
defined processes for continuous iteration and co-evaluation, shared contextual
data quality standards, and information scaffolds for both technical and
non-technical stakeholders to traverse expertise boundaries
An ADMM Based Framework for AutoML Pipeline Configuration
We study the AutoML problem of automatically configuring machine learning
pipelines by jointly selecting algorithms and their appropriate
hyper-parameters for all steps in supervised learning pipelines. This black-box
(gradient-free) optimization with mixed integer & continuous variables is a
challenging problem. We propose a novel AutoML scheme by leveraging the
alternating direction method of multipliers (ADMM). The proposed framework is
able to (i) decompose the optimization problem into easier sub-problems that
have a reduced number of variables and circumvent the challenge of mixed
variable categories, and (ii) incorporate black-box constraints along-side the
black-box optimization objective. We empirically evaluate the flexibility (in
utilizing existing AutoML techniques), effectiveness (against open source
AutoML toolkits),and unique capability (of executing AutoML with practically
motivated black-box constraints) of our proposed scheme on a collection of
binary classification data sets from UCI ML& OpenML repositories. We observe
that on an average our framework provides significant gains in comparison to
other AutoML frameworks (Auto-sklearn & TPOT), highlighting the practical
advantages of this framework
- …